Dynamic multi-scale loss optimization for object detection

With the continuous improvement of deep object detectors via advanced model architectures, imbalance problems in training process have received more attention. It is a common paradigm detection frameworks to perform multi-scale detection. However, each scale treated equally during training. In this paper, we carefully study objective detector We argue that loss level neither important nor independent. Different from existing solutions setting multi-task weights, dynamically optimize weight process. Specifically, propose an Adaptive Variance Weighting (AVW) balance according statistical variance. Then develop novel Reinforcement Learning Optimization (RLO) decide weighting scheme probabilistically makes better utilization without extra computational complexity and learnable parameters for backpropagation. Without bells whistles, proposed method improves ATSS by 0.9 AP on MS COCO benchmark. And it achieves 82.1 mAP Pascal VOC 2007 test set, which outperforms other reinforcement-learning-based methods.